feat(31): shared invariants module in quickwit-dst by g-talbot · Pull Request #6246 · quickwit-oss/quickwit

g-talbot · 2026-03-30T18:36:07Z

Summary

Adds invariants/ module to quickwit-dst — the single source of truth for invariant definitions shared across the verification pyramid (TLA+ specs, stateright models, production code)
InvariantId enum catalogs all 20 invariants (SS-1..5, TW-1..3, CS-1..3, MC-1..4, DM-1..5) with Display and descriptions
Shared pure functions: window_start_secs(), is_valid_window_duration(), compare_with_null_ordering()
check_invariant! macro always evaluates the condition (debug and release), implementing both Layer 3 (Prevention — debug_assert! panic in debug) and Layer 4 (Production — pluggable InvariantRecorder for Datadog metrics)
set_invariant_recorder(fn) wires up statsd at process startup; no recorder = no-op (single OnceLock load)
stateright is now an optional dependency behind model-checking feature — production crates pull only the invariant definitions, not the model checker
quickwit-parquet-engine uses shared functions and check_invariant! macro instead of inline debug_assert!

Stacks on gtt/phase-31-pg-metastore (PR #6245).

Test plan

cargo test -p quickwit-dst — 11 invariant unit tests + 2 doctests pass
cargo test -p quickwit-dst --features model-checking — all 24 tests pass (stateright models use shared functions)
cargo test -p quickwit-parquet-engine — all 153 tests pass
cargo test -p quickwit-metastore --all-features — all 187 tests pass (including postgres)
cargo test -p quickwit-indexing --all-features — 234 pass (8 pre-existing Pulsar/GCP PubSub flakes skipped)
cargo test -p quickwit-integration-tests --all-features — all 46 pass
cargo clippy -p quickwit-dst --all-features --tests — clean
cargo clippy -p quickwit-parquet-engine --all-features --tests — clean

🤖 Generated with Claude Code

fulmicoton-dd · 2026-03-31T09:44:34Z

quickwit/quickwit-cli/src/main.rs

+    quickwit_cli::logger::setup_dogstatsd_exporter(build_info)?;
+
+    #[cfg(not(any(test, feature = "testsuite")))]
+    quickwit_cli::logger::setup_invariant_recorder();


can we group those two into one thing?

Maybe call it:

quickwit_cli_logger::setup_metrics();

fulmicoton-dd · 2026-03-31T09:45:01Z

quickwit/quickwit-cli/src/logger.rs

+}
+
+#[cfg(not(any(test, feature = "testsuite")))]
+fn invariant_recorder(id: quickwit_dst::invariants::InvariantId, passed: bool) {


id -> invariant_id

Extract duplicated invariant logic into a shared `invariants/` module within `quickwit-dst`. This is the "single source of truth" layer in the verification pyramid — used by stateright models, production debug_assert checks, and (future) Datadog metrics emission. Key changes: - `invariants/registry.rs`: InvariantId enum (20 variants) with Display - `invariants/window.rs`: shared window_start_secs(), is_valid_window_duration() - `invariants/sort.rs`: generic compare_with_null_ordering() for SS-2 - `invariants/check.rs`: check_invariant! macro wrapping debug_assert - stateright gated behind `model-checking` feature (optional dep) - quickwit-parquet-engine uses shared functions and check_invariant! Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

The check_invariant! macro now always evaluates the condition — not just in debug builds. This implements Layer 4 (Production) of the verification stack: invariant checks run in release, with results forwarded to a pluggable InvariantRecorder for Datadog metrics emission. - Debug builds: panic on violation (debug_assert, Layer 3) - All builds: evaluate condition, call recorder (Layer 4) - set_invariant_recorder() wires up statsd at process startup - No recorder registered = no-op (single OnceLock load) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Emit cloudprem.pomsky.invariant.checked and .violated counters with invariant label via the metrics crate / DogStatsD exporter at process startup, completing Layer 4 of the verification stack. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

g-talbot requested review from fulmicoton-dd and mattmkim March 30, 2026 18:36

g-talbot mentioned this pull request Mar 30, 2026

docs: add CLAUDE.md, architecture docs, and /sesh-mode skill #6247

Open

4 tasks

fulmicoton-dd reviewed Mar 31, 2026

View reviewed changes

fulmicoton-dd approved these changes Mar 31, 2026

View reviewed changes

g-talbot force-pushed the gtt/phase-31-pg-metastore branch from c8836d6 to eccf657 Compare March 31, 2026 20:41

g-talbot force-pushed the gtt/phase-31-execute branch 2 times, most recently from 3eb3316 to c4418dd Compare March 31, 2026 20:55

g-talbot force-pushed the gtt/phase-31-pg-metastore branch 2 times, most recently from 6eeaecd to f2113e5 Compare March 31, 2026 21:03

g-talbot force-pushed the gtt/phase-31-execute branch 2 times, most recently from aa5ff45 to 142ddd6 Compare March 31, 2026 21:08

g-talbot force-pushed the gtt/phase-31-pg-metastore branch from f2113e5 to 0d561b4 Compare March 31, 2026 21:08

g-talbot force-pushed the gtt/phase-31-execute branch from 142ddd6 to 972bd20 Compare March 31, 2026 21:26

g-talbot force-pushed the gtt/phase-31-pg-metastore branch from 0d561b4 to 8ba9201 Compare March 31, 2026 21:26

g-talbot force-pushed the gtt/phase-31-execute branch from 972bd20 to 73c90c4 Compare March 31, 2026 21:29

g-talbot force-pushed the gtt/phase-31-pg-metastore branch from 8ba9201 to 0761d11 Compare March 31, 2026 21:30

g-talbot force-pushed the gtt/phase-31-execute branch from 73c90c4 to 44bd6bb Compare March 31, 2026 21:31

g-talbot force-pushed the gtt/phase-31-pg-metastore branch from 0761d11 to b551b13 Compare March 31, 2026 21:31

g-talbot force-pushed the gtt/phase-31-execute branch from 44bd6bb to add1902 Compare March 31, 2026 21:33

g-talbot force-pushed the gtt/phase-31-pg-metastore branch from b551b13 to 398cdf2 Compare March 31, 2026 21:33

g-talbot force-pushed the gtt/phase-31-execute branch from add1902 to d88f67f Compare March 31, 2026 21:40

g-talbot force-pushed the gtt/phase-31-pg-metastore branch 2 times, most recently from 3289f58 to dbbbd8b Compare March 31, 2026 21:50

g-talbot force-pushed the gtt/phase-31-execute branch from d88f67f to b5b840e Compare March 31, 2026 21:50

g-talbot force-pushed the gtt/phase-31-pg-metastore branch from dbbbd8b to 54a1f95 Compare April 1, 2026 10:48

g-talbot force-pushed the gtt/phase-31-execute branch 2 times, most recently from c8e6070 to cc704b5 Compare April 1, 2026 11:02

g-talbot force-pushed the gtt/phase-31-pg-metastore branch from 54a1f95 to 9696d99 Compare April 1, 2026 11:02

g-talbot force-pushed the gtt/phase-31-execute branch from bf93060 to cedd6ca Compare April 1, 2026 11:30

g-talbot force-pushed the gtt/phase-31-pg-metastore branch from bf23ab3 to a60b361 Compare April 1, 2026 11:30

g-talbot force-pushed the gtt/phase-31-execute branch from cedd6ca to 02cf95d Compare April 1, 2026 12:25

g-talbot force-pushed the gtt/phase-31-pg-metastore branch 2 times, most recently from dfa774f to 6a26333 Compare April 1, 2026 13:50

g-talbot force-pushed the gtt/phase-31-execute branch 4 times, most recently from c9634ed to 1df615d Compare April 1, 2026 15:34

g-talbot force-pushed the gtt/phase-31-pg-metastore branch from 25a1d84 to c4694ca Compare April 1, 2026 16:25

g-talbot force-pushed the gtt/phase-31-execute branch 2 times, most recently from 0c39228 to 3b1e48d Compare April 1, 2026 16:59

g-talbot force-pushed the gtt/phase-31-pg-metastore branch from c4694ca to 461fbb1 Compare April 1, 2026 16:59

g-talbot force-pushed the gtt/phase-31-execute branch from 3b1e48d to 251ec57 Compare April 1, 2026 18:18

g-talbot force-pushed the gtt/phase-31-pg-metastore branch from 13693e7 to 963fddb Compare April 1, 2026 18:36

g-talbot force-pushed the gtt/phase-31-execute branch 2 times, most recently from c4cbf0e to a96e8f5 Compare April 1, 2026 19:24

g-talbot force-pushed the gtt/phase-31-pg-metastore branch 2 times, most recently from b8c3a1d to 142e42a Compare April 1, 2026 20:18

g-talbot force-pushed the gtt/phase-31-execute branch from a96e8f5 to 9f71553 Compare April 1, 2026 20:18

g-talbot force-pushed the gtt/phase-31-pg-metastore branch from 142e42a to c23e666 Compare April 1, 2026 20:50

g-talbot force-pushed the gtt/phase-31-execute branch from 9f71553 to d7a7e05 Compare April 1, 2026 20:50

g-talbot force-pushed the gtt/phase-31-pg-metastore branch from c23e666 to ee0c5a4 Compare April 1, 2026 21:01

g-talbot force-pushed the gtt/phase-31-execute branch from d7a7e05 to 2c2eea2 Compare April 1, 2026 21:01

g-talbot and others added 4 commits April 1, 2026 17:13

fix: license headers + cfg(not(test)) for quickwit-dst and quickwit-cli

29cc07f

g-talbot force-pushed the gtt/phase-31-pg-metastore branch from ee0c5a4 to 605708e Compare April 1, 2026 21:14

g-talbot force-pushed the gtt/phase-31-execute branch from 2c2eea2 to 29cc07f Compare April 1, 2026 21:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(31): shared invariants module in quickwit-dst#6246

feat(31): shared invariants module in quickwit-dst#6246
g-talbot wants to merge 4 commits intogtt/phase-31-pg-metastorefrom
gtt/phase-31-execute

g-talbot commented Mar 30, 2026

Uh oh!

fulmicoton-dd Mar 31, 2026

Uh oh!

fulmicoton-dd Mar 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

g-talbot commented Mar 30, 2026

Summary

Test plan

Uh oh!

fulmicoton-dd Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

fulmicoton-dd Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants